Copyright © 1972 American Telephone and Telegraph Company 

The Bell System Technical Journal 

Vol. 51, No. 1, January, 1972 

Printed in U.S.A. 



Optimal Test Point Selection for 

Sequential Manufacturing 

Processes 

By M. R. GAREY 

(Manuscript received June 23, 1971) 

Consider a manufacturing process, such as the production of complex 
semiconductor devices, which consists of the sequential application of n 
possibly unreliable operations, d , t 2 , ■ ■ ■ , t n . Let c, be the cost incurred 
in performing operation t { , and let p, be the probability that t { will be 
performed successfully. Clearly one ivould prefer to reject immediately 
any item as soon as a faulty operation has been performed upon it in 
order to avoid the unnecessary cost of further processing that item. For 
this purpose, we shall assume that, immediately following each operation 
t { , it is possible to apply a perfectly reliable test T, , for determining 
whether or not the item should be rejected at that point, where the cost 
incurred by applying T { depends only upon the point i of test application 
and the last previous point at which such a test was applied. Since the 
application of tests entails additional costs, careful analysis is required 
to determine which tests are sufficiently useful to justify that additional 
cost. Using a dynamic programming approach, we derive a useful and 
efficient algorithm which utilizes the test and operation costs, along with 
the operation failure probabilities, to determine a set of testing points 
which will result in the minimal expected manufacturing cost. We then 
show how it is possible to further improve upon our algorithm for the 
particular case in which all test costs depend only upon the point of test 
application. 

Certain types of products, such as many semiconductor devices, 
are manufactured by performing a linear sequence of individual opera- 
tions, where many of the individual operations have a nonnegligible 
probability of yielding an undesired result. In such cases it would be 
preferable to immediately reject a faulty item to avoid the unnecessary, 
and useless, cost of further processing that item. However, the in- 
troduction of tests into the manufacturing process, for determining 
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whether or not an item should be rejected, will itself introduce ad- 
ditional costs in such a way that the total expected cost of the manu- 
facturing process may increase. In addition, certain choices of test 
points, though reducing the total expected cost, may be considerably 
more costly than certain other choices of test points. We shall present 
an efficient and useful algorithm that uses the various test and operation 
costs, along with the operation failure probabilities, to determine an 
optimal set of testing points, which minimizes the expected cost of 
the entire manufacturing process. 

Consider a manufacturing process consisting of n individual opera- 
tions, or tasks, U , U , ■ • • , t n , which are to be performed sequentially 
in the fixed order of their indices. Associated with each task t t are the 
cost c, > incurred by performing *, and the success probability p } - , 
< p, ^ 1, that operation tj will have the proper result, assuming 
all previous operations were successful. Immediately after any opera- 
tion tj it is possible to perform a perfectly reliable test T, which deter- 
mines whether or not all previous operations, including % % , have been 
successful, that is, whether or not the item should be rejected at that 
point. The cost for performing test T, is given by C, > 0. We justify 
this simple testing cost by noting that in many manufacturing pro- 
cedures of this type the major determinant of testing cost is merely 
the cost of removing the item from the assembly line and preparing 
it for test application. However, we shall later generalize the testing 
cost to allow a dependence upon the previously untested operations. 
We now develop an algorithm which uses the knowledge of these costs 
and probabilities to determine a choice of test points (a subset of the T t ) 
which results in the lowest possible expected manufacturing cost per 
item processed. This is equivalent to minimizing the expected cost per 
fault-free item produced, because the introduction of tests into the 
process does not change the probability, given by II"=i P> > tnat an 
item is successfully processed by all n tasks. The tests merely serve to 
reject already faulty items to avoid the cost of useless additional 
processing. 

We solve this problem by the method of dynamic programming.' 
Following each operation i, it is necessary to decide whether or not 
test Tj should be applied. To make this decision we need only compute 
the minimal expected costs for completing the process assuming that 
Tj is or is not utilized and choose the smaller. The computations of 
these two expected costs use the cost of T, , the probability p that the 
item is not faulty at this point, and the cost of t i+1 . For the case that 
Tj is used, we also need the minimal expected cost for completing the 
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process under the assumption that the item is fault-free before t i+l 
is performed. For the case that T, is not used, we need the minimal 
expected cost for completing the process under the assumption that 
the item is not faulty with probability p before t i + t is performed. 

Formally, let a state of the process be described by [j, p], with 
1 ^ j ^ n and < p £ 1, where the process is in state [j, p] if and 
only if operation t t has just been applied and the resulting item is 
fault-free with probability p. There is also a "reject" state R entered 
when a test rejects a faulty item. The initial state of the process is 
[1, pj, and the final states are all those states of the form [n, p], 
< p ^ 1, plus the reject state R. The transitions between states 
are determined by the decisions on whether or not to apply a test 
before the next operation. If the process is in state [;', p] and test T, is 
not applied, the next state will be [j + 1, p-p,- +I ]. If the process is in 
state [j, p] and test T, is applied, the next state will be either [j + 1, p i+l ], 
with probability p, or R, with probability 1 — p. The problem is to 
find a sequence of states to transform the process from the initial state 
to any final state, with minimal expected total process cost. 

Let K[j, p] be the minimum expected cost to get from state [j, p] 
to a final state and let S[j, p] be the corresponding optimal set of test 
points T k . We then have 

m P ] = min{ C -+^' + ^ +1 - ?•*']> 
\c i + 1 + K[j + l,pp, +1 ] 

and S[j, p] equals either S[j + 1, p l+1 ] W [T,\ or S[j + 1, />•/;, + ,], 
depending upon whether the first or second expression is smaller. For 
a final state [n, p], < p ^ 1, we have K[n, p] = and S[n, p] empty. 
(In our basic model, the final test will never be applied, since it can 
only increase the total expected cost. However, the results can also 
be applied if test T n is always required as a final product quality check, 
in which case K[n, p] = C n and S[n, p] = {T n }.) 

The difficulty with using these equations to obtain an optimal 
solution is that, since p can take any value from to 1, there are an 
infinite number of equations to solve. Fortunately, for states of the 
form [j, p], p can take only a finite number of possible values, XT*=i P* > 
IIft-2 P* , • • • , TL'k-j P* i depending upon the last test which had been 
applied. Thus, if none of the tests T, , T i+1 , • • • , 7V, were applied, 
and either i = 1 or 7\_, was applied, then p = YU-i Vk • Accordingly 
we shall now describe a state of the process by (;', i), i ^ j, where the 
process is in state (j, i) if and only if 
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(?) operation t, has just been applied, 

(ii) none of the tests 7\ , T i+1 , ■■■ , T,_, were applied, and 
(tit) either i = 1 or T,_, was applied. 
The initial state is then state (1, 1) and the final states are of the form 
(ft, f), 1 ^ i ^ n, plus the reject state R. If the process is in state 
(j, i), the next state will be either (j + 1, j + 1), with probability p, 
or R, with probability 1 - p, if T, is applied or + 1, i) if 7\ is not 
applied. With K(j, i) and S(j, i) defined in the same way as previously, 
the dynamic programming equations become 



K(j, i) = min 



Ci - + (llp*)-(c, + i + K(j + l,i+ D) 



fit+i + K(j+ l,i) 
and S(j, z) equals either S(j + 1, j + 1) U [T,\ or S(j + 1, t), de- 
pending upon whether the first or second expression is smaller. We 
also have K(n, i) = and S(n, i) empty for 1 ^ i ^ n. This set of 
equations can be solved iteratively to determine the optimal solution. 
Observe now that no additional complications arise if we generalize 
the test costs to allow a dependence upon the state (j, i). Accordingly 
we redefine the cost of test T, , when the previously untested operations 
are t { , t i+1 , ■■■ , t, , , to be given by C,., > 0, allowing the additional 
dependence upon i. We can then rewrite our dynamic programming 
equations as 

vr . . |c,., + (np*Wi+TO+i,j+i)) 

K(], i) = min J \r=i / 

.c, + . + #0' + 1,0 
with JC(n, t) = for 1 ^ i ^ n, with S(j, i) corresponding as before. 
The following algorithm uses these equations to iteratively solve for 
the optimal set of test points. 

Algorithm 1. Computation of Optimal Testing Points 

(a) For each i, 1 ^ i ^ n, set 

K(n, i) <- and S(n, i) <- (empty set). 

(b) Set j <- n - 1. 

(c) Set i <- j and P 4- 1. 



(d) Set P*-P-p, . 

(e) Compute 

K(j, i) = min 



C i . i +P-(c i+1 + K(j+l,3+ D) 
c, +1 + K0' + 1, i) 
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and set S(j, i) to S(j + 1, ; + 1) U [T t ] or S(j + 1, i), de- 
pending, respectively, on whether the first or second expression 
is smaller. 

(f) If i > 1, set i <— i — 1 and go to (d). 

(g) If j > 1, set j *— j - 1 and go to (c). 

(h) The optimal set of test points is given by S(l, 1) and yields a 
manufacturing process with total expected cost per item processed 
ofc, +K(1, 1). 

Notice that it is not necessary to actually construct and save all 
the optimal partial sets S(j, i) but merely to note whether or not test 
Tj is included in S(j, i). The optimal test set can be constructed directly 
from this information upon completion of the algorithm. The number 
of operations performed by the algorithm is at most proportional to 
the square of the number of tasks in the manufacturing procedure 
under analysis. Thus the algorithm can be economically used for 
extremely large problems when programmed for a digital computer, 
and many problems can be reasonably solved by hand calculation. 
The details of the algorithm are illustrated with a small example in 
Appendix A. 

Notice that if physical constraints prevent the application of a test 
following certain tasks, this can be handled in the algorithm merely 
by automatically setting K(j, i) to c,+, + K(j + 1, ?') whenever T, 
is not allowed. This is equivalent to combining tasks t, and t l + i into a 
single task having cost c, + c l+ i and success probability prp,+i • 

We now return briefly to the original case where the cost of test T, 
depends only upon ;' and show how Algorithm 1 can be improved for 
this case. We let 

D(j, i) = C, + (n p k )(c i+l + K(j + 1, j + 1)) - c, + l - K(j + 1, i). 

Thus D(j, i) is merely the difference of the two expressions which must 
be compared in order to determine whether or not to apply test 7 1 , 
when in state (;', i). An optimal solution is formed by choosing to apply 
test Tj from state (j, i) if and only if D(j, i) < 0. We can then prove 
the following theorem. 

Theorem 1: For all j, 1 < j < it, and all i, 1 < i < j, D(j, i) 
^ D(j, i - 1). 

The proof of Theorem 1 is given in Appendix B. Simply stated, 
Theorem 1 tells us that, for fixed j, the difference function D(j, i) 
changes sign only once. In particular, it implies that if test T t is to be 
applied in state (j, i), then test T, should also be applied in each of the 
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states (j, i - 1), (j, » - 2), ■ • • , 0*, 1)- This enables us to modify step 
(e) of Algorithm 1, when all test costs depend only on j, to omit the 
computation of c, + 1 + K(j + 1, i) in determining K(j, i) whenever 
Tj t S(j, i '+ 1). In fact, for very large problems, one might consider 
using a binary search, for each j, to discover the value i = i„ at which 
D(j, i) changes sign. Once that point has been found, each K(j, i) 
can be computed by evaluating only one of the two expressions in the 
minimization, the proper one being determined by whether i is less 
than i or greater than ?'„ . Though the number of operations required 
will still increase proportionally to n 2 , this method will substantially 
reduce the actual number of operations performed. 

A related problem with tests which merely test for the success or 
failure of a single operation, all of which must be applied, but which 
are not restricted as to point of application in the process sequence is 
solved as a special case in Ref . 2. The algorithm given there requires time 
bounded by the square of the number of tasks in the production 
sequence. A more general problem has tests which each determine the 
success or failure of some particular subset of the tasks. Any one of the 
tests may be applied at any point in the manufacturing process and 
will determine the success or failure of all operations which belong to 
its test set and which have already been performed. The problem solved 
in this paper had those test sets restricted to being of the form {t t , 
U , ■ ■ ■ , h] i which enabled us to give an efficient optimization algo- 
rithm. The best known algorithms for the general problem, however, 
require an amount of time which may be exponential in the number 
of tasks, restricting their usefulness to only very small problems. 

Another related problem has been considered by T. S. Ellington. 3 In 
his problem, one has a number of alternative methods for accomplishing 
each task, the alternatives having different cost and success probabilities, 
and one would like to select the best method for each task in order 
to obtain the minimum expected cost. Though Ellington was able to 
give an efficient algorithm for making this selection, it was based on 
the assumption that each task is automatically tested immediately 
after it is applied. If, however, we also allow choice as to test points, 
the problem becomes more complicated. One cannot merely combine 
our algorithm with Ellington's, though both are dynamic programming 
algorithms, because they work from opposite ends of the process 
sequence. The development of an efficient algorithm to simultaneously 
select optimal test points and optimal task alternatives remains an 
open problem which could have substantial practical importance in 
the design of actual manufacturing processes. 
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APPENDIX A 

Optimization Algorithm Example 

We illustrate the application of the optimization algorithm with 
the following small example: 

d pi C, 
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If no tests are applied, the expected cost of the process is simply 
the sum of the c, or 110. If all tests, except the last (which can only 
increase the total expected cost), are applied, the expected cost of the 
process is 

C, + C, + p t (c 2 + C 2 + p 2 -(r, + C, + 7V(C4 + C 4 + p 4 

•(c a + C 5 + p s -(c)))))^ 101.77. 

We now show the successive computations performed in applying the 
optimization algorithm. 

A76, i) = for all i no test 

K(5, 5) = min (10 + .8(15), 15) = 15 no test 

7C(5, 4) ='min (10 + .64(15), 15) = 15 no test 

K(5, 3) = min (10 + .576(15), 15) = 15 no test 

K(r>, 2) = min (10 + .4008(15), 15) = 15 no test 

K('), 1) = min (10 + .41472(15), 15) = 15 no test 

K(4, 4) = min (12 4- .S(45), 45) = 45 no test 

X(4, 3) = min (12 + -72(45), 45) = 44.4 test 

K(A, 2) = min (12 4- .570(45), 45) = 37.92 test 

K(A, 1) = min (12 4- .5184(45), 45) = 35.328 test 

/v(3, 3) = min (4 + .9(65), 64.4) = 62.5 test 

K(S, 2) = min (4 + .72(65), 57.92) = 50.8 test 

/C(3, 1) = min (4 4- .048(65), 55.328) = 46.12 test 
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K(2, 2) = min (8 + .8(87.5), 7.5.8) = 75.8 no test 

K(2, 1) = min (8 + .72(87.5), 71.12) = 71 test 

K(l, 1) = min (7 + .9(85.8), 81) = 81 no test 

These all are computed by simply following the optimization algo- 
rithm as given, noting by "test" or "no test" whether the first or second 
value was smaller. The optimal solution has expected cost c, + 81 = 91, 
which is appreciably smaller than the two "obvious" solutions. We 
remind the reader that the cost per fault-free item produced is obtained 
simply by dividing the cost per item processed by Xl"=i P< > the prob- 
ability that the processed item is fault free. Thus, in this case, the 
optimal solution has cost per fault-free item approximately equal to 
233, compared to 282 if no tests are used and 261 if all tests are used. 
Considerably greater benefits may be derived when the algorithm is 
applied to processes with a larger number of tasks. 

To determine which tests are applied in the optimal solution, one 
notes first that 7\ is not applied since "no test" was indicated for 
K(l, 1). Since T, is not applied, we go to K(2, 1) which indicates that 
T 2 should be used. Since T 2 is applied, we go to K(S, 3) which indicates 
that T 3 should be used. Since T 3 is applied, we go to K(4, 4) which 
indicates that T 4 should not be applied. Next, since T 4 is not used, we 
go to K(5, 4) which indicates T-, is not used. Finally, K(Q, 4) indicates 
that T a is not used. Thus, the optimal solution uses only two tests, 
T 2 and T 3 . 

If one is required to apply T 6 , the optimal solution can be computed 
similarly to have expected cost 98.6. It uses only tests T 2 , T 4 , and 
T a . In this case, the solution which uses no tests other than T has 
expected cost 122, and the solution which uses all the tests has expected 
cost 106.75. We emphasize again that, even though the gain from 
applying the optimization algorithm to this small example is not 
insubstantial, considerably greater savings can be obtained with larger, 
more realistic problems. 

APPENDIX B 

Proof of Theorem 1 
Let 

D(j, i) = C, + (il p*)(c, +1 + K(j + 1, j + 1)) - c, +1 - K(j + 1, i). 

Theorem 1: For all j, 1 < j < n, and all ?', 1 < i ^ j, D(j, i) ^ D(j, 
i - 1). 
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Proof. Suppose D(j, i) < D(j, i - 1). Then 
D(j, i - 1) - D(j, i) = (n^)(P-i " l)fc,« + K(j + 1, ; + 1)) 

+ tf(;+ l,t) - K(;+ 1,»- 1) > 0. 
Then 
K{j+ \,i) - K(j+ l,i- 1) 

> (n^)(l - P,-i)(c 1+1 + K(; + 1, j + 1)). 

We shall show that this inequality is never satisfied; that is, for all 
;*, 2 < ; ^ n, and for all i, 1 < i ^ j, 

(*) K(j, i) - K(j, i - 1) £ (fl P*)(l - P.-.)(c, + tf(j, j)). 

We prove (*) by induction on j, running from n down to 3. If j = n, 
we have that K(n, i) = K(n, i — 1), so that the left side of (*) is zero. 
Since the right side is nonnegative, the inequality is clearly satisfied. 

Now suppose that (*) holds for all j, m < j ^ n, and all i, 1 < i ^ j. 
We shall show that it must then hold for j = m. 

To do this, we first prove a useful intermediate inequality, 

p m (c m+l + K(m + 1, m + 1)) ^ c m + K(m, m). 

We consider two cases. If D(m, m) < 0, 

c m + K(m, m) = c m + p m (c m+I + K(m + 1, m + 1)) 

^ p m (c m „ +K(m + l,w + 1)). 

If D(m, m) ^ 0, c m + K(w, »n) = c m + c m+1 + tf(??? + 1, m). By the 
induction hypothesis, we have 

K(m + 1, m + 1) - #(»? + 1, m) 

< (1 - p M )(e B+] + K(m + l,«i + 1)). 

Rewriting, we obtain 

c m+l K(m + 1, m) ^ p ni (c m+1 + K(m + 1, m + 1)), 

which completes the proof of the intermediate inequality. 

We now prove (*) for j' = m, using two cases. 

Case 1. D{m, i - 1) < 0. Since D{m, i - 1) < 0, 

K(m, i - 1) = C m + ( n P>)(Cm +i + K(m + 1, m + 1)). 
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It is always the case that 

K(m, i) ^C m + (ft p 4 )(c m+1 + K(m + 1, m + 1)). 

Thus, 
K(m, i) - K(m, i - 1) 

^ (n^) 1 - P— i)(c- + i + K(m+ 1, m + 1)) 

= (H^a ~ Pm-r)[p m (c m+l +K(rn + l,m+ 1))] 

^ (l(^)(l - P«-0[c- + K(m, m)], 

using the intermediate inequality proved earlier. This completes the 
proof of (*) for Case 1. 

Case 2. D(m, i - 1) ^ 0. Since D(m, i - 1) > 0, 

K(m, i - 1) = c m+1 + #(m + 1, i - 1). 

It is always the case that K(m, i) ^ c m+I + K(m + 1, i). Thus, tf(m, i) 
- K(m, i - 1) ^ tf(m + 1, ») - K(m + 1, * - 1). By the induction 
hypothesis, 

K(m+ l,i) - K(m + 1, i - 1) 

^ (ftp*)(l - P.-i)(c» + i + £(m + 1, m + 1)) 

= (]IP*)a - P«-i)[p-(e.« + X(m + 1, m + 1))] 

^ (n^-)(l - P.-i)[c« + K(m, m)], 

again using the previously proved intermediate inequality. This com- 
pletes the proof of (*) for Case 2. 
Theorem 1 follows by induction. 
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